Inference for Normal Mixtures in Mean and Variance

A finite mixture of normal distributions, in both mean and variance parameters, is a typical finite mixture in the location and scale families. Because the likelihood function is unbounded for any sample size, the ordinary maximum likelihood estimator is not consistent. Applying a penalty to the likelihood function to control the estimated component variances is thought to restore the optimal properties of the likelihood approach. Yet this proposal lacks practical guidelines, has not been indisputably justified, and has not been investigated in the most general setting. In this paper, we present a new and solid proof of consistency when the putative number of components is equal to, and when it is larger than, the true number of components. We also provide conditions on the required size of the penalty and study the invariance properties. The finite sample properties of the new estimator are also demonstrated through simulations and an example from genetics.